A Novel Method to Estimate the Model Uncertainty Based on the Model Errors

This paper presents a novel method for estimating “total” predictive uncertainty using machine learning techniques. By the term “total” we mean that all sources of uncertainty are taken into account, including that of the input and observed data, model parameters and structure, without attempting to separate the contribution given by these different sources. We assume that the model error, which is mismatch between the observed and modelled value reflects all sources of uncertainty. Fuzzy c-means clustering was employed to cluster the input space into different zones or clusters assuming that the all the examples those belong to the particular cluster have similar model errors. The prediction interval is constructed for each cluster on the basis of empirical distributions of the historical model errors associated with all examples of the particular cluster. Prediction interval for the individual example is derived from cluster based prediction interval according to their membership grades in each cluster. Linear or non-linear regression model is then built in calibration data that approximates an underlying functional relationship between an input vector and the computed prediction intervals. Finally, this model is applied to estimate the prediction intervals in verification data. The method was tested on hydrologic datasets using various machine learning techniques. Preliminary results show that the method has certain advantage if compared to other methods.